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MEMORANDUM FOR: 


SUBJECT: Participation in DELPHI Experiment 


* 

1. One of the recent developments in long-range planning is 
the so-called DELPHI Technique, i.e., the systematic use of expert 
opinion for obtaining answers and, if possible, a consensus from a 
panel of experts on a particular issue. We have received approval 
from the Director, PPB to test the DELPHI Technique and have 
selected Career Personnel Management as the subject for a pilot 
study. 


2. Briefly, we would like you to participate in this experiment 
as one of the study's experts. Note well that the attached experiment 
is exactly that -- an experiment, and participation is strictly voluntary. 
If you agree to participate, the overall demands on your time will be 
approximately 6-9 hours spread over several weeks, i.e., 1 to 2 
hours per questionnaire. More time will be required at the outset, 
to become acquainted with the experimental ground rules, than in 
the latter stages of the DELPHI questionnaire. 


3, Our version of the DELPHI employs the anonymous method 
of response, i,e,, respondents will be UNKNOWN to each other and 
known only to a 3-man DELPHI Panel for identification, scoring, and 
feedback. 


4, Let us know by 6 February if you can participate or not 
participate. One of the members of the scoring panel will be around 
to discuss this matter with you and answer any questions you may 
have on the nature of the DELPHI and/or the questionnaires. 
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EXPERIMENT IN HUMAN 
INFORMATION PROCESSING 


DELPHI I 
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PREFACE TO INSTRUCTIONS 


The experiment in which you are taking part is one ofa 
series investigating human information processing, Your primary 
task will be to answer questions of a specific information type. The 
experiment is not a quiz or an examination, nor is it a test of social 
influence. We are interested in the way groups use incomplete 
information to arrive at factual conclusions. 

It is not expected that y,ou will know the exact answer to any 
of the questions. However, for most of them you will have some 
general knowledge that will enable you to make an estimate--an 
' informed guess--of the answer. You are to make as good an 
estimate as you can; but in any case, answer every question as 
best you can. 

On page 3 is the Respondent's Form which requests certain 
background information. As in any experimental study, certain 
(and in this case minimal) descriptive data are required. 

This experiment, investigating career management, is nota 
hidden and manipulative process by which Respondents are brain- 

’ washed into thinking, believing, and feeling the way someone might 


want them to without realizing what is happening. The process will 
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be open in that it will be continually described and discussed (via 
feedback between rounds) with the participants. Moreover, the 
experiment is not guided by a staff panel which is covertly in 
control and by some magic hides this fact from the Respondents. 
One objective of this experiment is not to suppress conflicting 
answers to the questions. Consequently, all Respondents will be 
anonymous and each respondent will be assigned a number for 
identification purposes only. In order to maintain anonymity do 
not discuss the questions or the experiment with any of known or 
suspected participants. The panel will be available at any time if 
any questions concerning your task should arise. It should be 
- noted that a copy of your answers should be retained for reference 
and comparison in the succeeding rounds and answers to questions 
should be typed before returning them to the panel. Limit your 
response to each question requiring a narrative answer to one 
page. Additionally, if the list of given questions motivate you to . 
suggest subsidiary questions, write them ona separate sheet and 
append them to the questionnaire. The time requirement to complete 
the first questionnaire is 2 weeks. If for some reason you cannot 
complete your questionnaire within the two-week time period, 


contact the panel for special arrangements. 
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INSTRUCTIONS 


This is the first ina series of several Rounds (perhaps as 
many as 5), intended to obtain from your group as accurate an 
estimate as possible of the answer to each of the questions listed 
ona separate sheet. The subject for this DELPHI experiment is 
Career Management. 

Before attempting to answer any of the primary questions, 
read all of them, and rate ery. with regard to your relative 
confidence in contributing to their answers. Specifically, pick 
out those questions where you feel relatively most confident to 
eenieieies to its answer and write '5' next tothe answer. Next, 
pick out that statement where you feel relatively least confident 
to contribute and write '1' next to the answer, Having thus 
established a scale from 1 to 5 within which to express your 
relative confidence, rate yourself with regard to the remaining 
questions by similarly writing '1', '2', '3', '4', or '5' next to 


the appropriate statement. 
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Some of the questions call for a numerical answer. To the 
right of the question are three blanks, These are to be filled in 
with numbers which represent a low estimate, a mid estimate, 
and a high estimate. As anexample, suppose the question were: 

What is the present temperature of this room? 
L____ degrees 
M 


ee 


H 


The temperature is certainly no lower than 60 degrees and certainly 


no higher than 85 degrees. Thinking along the lines of the figure 


Low Mid High 
60 | 7 | 85 
68 73 78 


as you proceed up the scale, you might judge that around 68 degrees 
there is a one to three chance that the temperature is lower. That 
would be your low estimate. At around 73 degrees you might think 
there is a 50-50 chance that it is higher--your mid estimate. 
Finally, you might consider that at 78 degrees the chances are no 
better than one in three that the temperature is higher and that would 


‘be your high estimate. The numbers just stated are only illustrative, 


ao 
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The questions you are being asked in the experiment are not 
concerned with simple physical quantities like temperature, 


but the general procedure of formulating low, mid, and high 


estimates is the same. Questions requiring a numerical answer 
are flagged with an asterisk. In the first Round, please answer 


the questions relying on what background information you may 
have. Do not look up or research the answers, but be impression- 
istic. We ask your cooperation on this experiment rule. In the 
second and subsequent Rounds you will be furnished a summary 

of the answers for the group. The Summary will consist of the 

. median answers (when possible) of the group, and the two quartiles, 
that is, the range in which 50% of the group's answers are found. 
This summary is a form of "pooling" of the information of all 

the members of the group and will serve as a basis for revising 


your answers if it seems appropriate. 
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Questionnaire No. 1 


On the following pages there are several questions of 
a specific information type. Sore will require a 
numerical answer (flagged with an asterisk) while 
others will require a narrative. Remember--keep 


a record of your responses. 
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DELPHI I 


QUESTIONNAIRE #2 
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QUESTIONNAIRE #2 


PART I. 


This is the second in our series of several DELPHI questionnaires. 


The first seven questions that had been posed in the first questionnaire 


(with a few modifications) are repeated below, together with the infor- 


mation on the median and the interquartile range (IQR) of the 15 first 
round responses, [The IQR is the interval containing the middle 50% 
of the respons es. / The average group rating is given as a matter of 
information only. You need not re-rate yourself on this round. 

Please reconsider your previous estimate, change it if you 
wish: ! (only your best estimate for Round 2). Whenever your present 
Round 1 answer (i.e., mid-estimate) is OUTSIDE the interquartile 
range (IQR), briefly state your reason why you think the answer should 
be greater (or less) than the IQR than that given by the majority of the 
respondents. ! (No such reason needs to be given when your answer is 


inside the JOR.) 


las in Round 1, keep a copy of your answers and please type your 
returned answers. 


1 
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QUESTIONNAIRE #2 (Continued) 


PART II. 

In this part, the same nine questions (8 through 16) that had 
been posed in the first questionnaire are repeated below, together 
with the reasons ''For, Acseek, Don't Know, etc."' 


A. Please reconsider your previous answer, this time 


taking into account the information offered by other respondents 


on the preceding round. Consider the stated reasons, give 
them whatever weight you think they deserve, and then write 
your present (possibly revised) estimate as to the answer to 
the question. | (If same as before, so state. ) 

B. Also provide a "Critique of Reasons", i.e., briefly 
state which one of the reasons offered by others on each 


question you found unconvincing and why.! 


tas in Round 1, keep a copy of your answers and please type your 


.returned answers. 
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